1. Technical Field
This invention relates to monitoring computer implemented instructions. More specifically, the invention relates to user-space probes and storage of a breakpoint instruction in user space for use in monitoring returns from subroutines.
2. Description of the Prior Art
An operating system segregates memory into kernel space and user space. The kernel is the central module in most operating systems that manages the system's resources and communication between hardware and software components. It is the part of the operating system that loads first and remains in main memory. The kernel is responsible for many tasks, including memory management, process and task management, and disk management.
Kprobes, a kernel probe (kprobe) is a simple and lightweight mechanism that allows one to gather additional information about kernel operation without recompiling or rebooting a kernel. Kprobes enables a location in the kernel to be instrumented, such that an instrumentation subroutine runs when the processor encounters that kernel probe. Once the instrumentation subroutine completes execution, the kernel continues normal execution.
When investigating a problem in the kernel or a user program, various types of diagnostic tools may be used. Examples of such diagnostic tools include source-code modifications to log diagnostic messages, interactive debuggers, and dynamic, non-interactive probing systems. This invention pertains to a dynamic, non-interactive probing system. With such a system, instrumentation can be created, installed, and removed dynamically while the instrumented program runs at or near full speed. Examples of such systems include IBM's Dprobes for the OS/2 and Linux operating systems, Sun Microsystems' Dtrace for Solaris, and IBM's Kprobes (kernel probes) and Uprobes (user-space probes) for Linux. In Dprobes and Dtrace, an instrumentation subroutine is written in a special language, and is run on an emulator; this emulator is part of the kernel. In Kprobes and Uprobes, the instrumentation subroutine is written in the same language as the kernel and runs directly on the processor. More specifically, in Kprobes-based instrumentation is built as a kernel module. So, rather than having to recompile and reboot the system with an instrumented kernel, a Kprobes instrumentation module can be written, compiled, and loaded on the system. There is no need to reboot the system. The Kprobes instrumentation module inserts a kprobe at a specific instruction. More specifically, Kprobes replaces the specified instruction, or the first byte(s) thereof, with a breakpoint instruction. The effect of the breakpoint instruction, when executed, is to save certain of the processor's registers and transfer control to a breakpoint-handler subroutine in the kernel. This subroutine passes control to Kprobes. The instrumentation provided by a kprobe could be inserted anywhere in a subroutine. In one embodiment, the kprobe is used as a reporting mechanism to help determine the location of a bug in the kernel.
A kernel return probe (kretprobe) is a feature of Kprobes that allows instrumentation to be placed at the return point of any non-inline function in the kernel. In the kretprobe feature, Kprobes needs to gain control of the central processing unit when the probed subroutine returns. This control is accomplished by replacing the return address of the probed subroutine with the address of a piece of code, also known as a trampoline. A trampoline is a small piece of code used to redirect the flow of execution in an unusual way. A trampoline may be a sequence of instructions, or simply an address; the trampoline may be created statically or on the fly, and it may be used to transfer control between kernel and user mode. With respect to a kretprobe trampoline, the trampoline may be a single instruction or a sequence of instructions, depending on the architecture. It's created statically, and transfers control within kernel mode. In the following example of a kretprobe, the trampoline transfers control to Kprobes. FIG. 1 is a block diagram (100) showing how a kernel return probe functions. Return probes work by hijacking the return address upon entry to the probed function. Assume that a return probe has been registered on kernel function bar( ) (102). As a result, a kprobe is registered at the entry to bar( ). When foo( ) (104) calls (110) bar( ) (102), the processor pushes the return address in foo( ) (104) onto the stack, or stores it in a special register, and jumps to bar( ) (102). The entry point kprobe is hit and Kprobes gains control. Kprobes saves a copy of the return address and replaces the return address in the stack, or in the special register, with the address of the aforementioned kretprobe trampoline (106). When the function bar( ) (102) returns, the effect is a jump (112) to the trampoline (106) in place of a normal return (108). The trampoline contains a breakpoint instruction, or other code that transfers control to Kprobes. Kprobes runs the user-specified instrumentation subroutine (not shown) associated with that kretprobe and then jumps to the original return address (120).
As noted above, the operating system includes both user space and the kernel space. User space is the memory area where all user mode applications run. A user application cannot access kernel space directly. There are user-space equivalents of kernel probes and kernel return probes, and these are known as user probes (uprobes) and user return probes (uretprobes), respectively. For a uretprobe, the trampoline can consist of a single breakpoint instruction. A problem arises with the uretprobe as to where to place the breakpoint instruction in the process's address space. Any solution that requires modification of the source code of the user application is unacceptable because it defeats the purpose of the user-return-probe feature (dynamic instrumentation). The breakpoint instruction cannot be placed in any portion of the user process's address space that might be overwritten during execution of the probed function. Therefore, it is not practical to place the breakpoint in the stack or data section of the process's address space.
Accordingly, there is a need for locating the trampoline breakpoint instruction of a user-space return probe in a location in user space that already exists in the process's virtual address space, will not be overwritten during execution of the probed function, and will be uniform across different applications.